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ABSTRACT 



This document introduces a new school accountability 
mechanism, the Accountability Dialogue. An Accountability Dialogue brings 
together educators, parents, and the larger school community to: (1) 

establish standards for school and student performance; (2) determine how the 
school is doing in relation to those standards; (3) investigate the 
conditions that contribute to the school's strengths and weaknesses; and (4) 
plan how to address the areas where improvement is needed. The Western 
Assessment Collaborative worked for 5 years with a set of districts and 
schools in their efforts to implement standards-based reform. By drawing on a 
multiplicity of perspectives and sources of data, Accountability Dialogues 
acknowledge, rather than ignore, each of the challenges raised by large-scale 
external accountability systems. At the same time that Accountability 
Dialogues accelerate the implementation of standards and facilitate the use 
of data to plan improvements, they also upset the traditional ways in which 
educators have related to parents, the community, and one another. The 
greatest promise of Accountability Dialogues lies in their potential to 
strengthen the relationships that are essential to sustained school 
improvement. Appendix A contains a checklist for planning and conducting 
Accountability Dialogues, and Appendix B contains overheads to introduce 
concepts of accountability and Accountability Dialogues. Appendix C contains 
materials for assisting participants to look at student performance data. 
(Contains 12 references.) ( SLD) 
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he call for greater school accountability will not be ignored. Journalists, politi- 
J cians, parents, educators, policymakers, and the public are all weighing in on 
the theme. Presidents and governors are elected on promises to sanction low- 
performing schools and reward successful ones. Mayors promote former military 
leaders to run their schools, and new superintendents earn their stripes by 
promising to “clean house” and rid the system of ineffective school leaders. 
States threaten to take over low-performing school districts, districts to reconsti- 
tute or reassign the staff of low-performing schools, and schools to withhold 
diplomas from or retain students deemed unsuccessful. Top-down accountability 
sends a clear message — “someone must pay.” 

Not surprisingly, this call for accountability has created considerable unrest 
in the educational community. Critics of the public school system attribute this 
unrest to what they see as the long-standing unwillingness of educators to accept 
responsibility for the quality of schools and student performance. Many educa- 
tors, on the other hand, acknowledge the need for greater accountability, but 
question the fairness of many recently enacted plans and caution against their 
unintended consequences. Recently adopted plans reinforce the idea that 
accountability is something “done to” educators rather than a value sustained 
from within the system and by the individuals who work there. 

A New School 
Accountability Mechanism 



'his document introduces a new school accountability mechanism, the 
J Accountability Dialogue. An Accountability Dialogue brings educators, par- 
ents, and the larger school community together to establish standards for school 
and student performance, determine how the school is doing in relation to those 
standards, investigate the conditions that contribute to the school’s strengths and 
weaknesses, and plan how to address those areas where improvement is 
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Assessment for ACCOUNTING 

“ Accounting refers to the act of gathering, organizing, and making 
available for use any of a variety of information describing the 
performance of the system ” 

Assessment for ACCOUNTABILITY 

“ Accountability takes that information and uses it to inform judgements 
about performance and how it can be improved — including planning 
and action.” 

ACCOUNTABILITY DIALOGUES 

“The goal is to involve various constituencies in the construction of deep 
and commonly held understandings about the performance of the system, 
the reasons for the performance, the best possible ways of improving 
performance, and the responsibilities of each in doing so.” 

- Paul G. LeMahieu, " From Authentic Assessment to Authentic Accountability ” 



needed. Accountability Dialogues are forums through which schools and the 
larger community strive to become partners in the effort to assure that eveiy stu- 
dent achieves to high standards. The dialogue is conducted on the premise that 
this process of professional and public engagement will help schools develop a 
sense of internal accountability that will guide continuous improvement. 

Under the auspices of the Regional Educational Laboratoiy at WestEd and 
with generous support from the Stuart Foundation, the Western Assessment Col- 
laborative (WAC) has worked for five years with a set of districts and schools in 
their efforts to implement standards-based reform. Accountability Dialogues have 
served an important role in these efforts. 

In supporting schools to conduct Accountability Dialogues, WAC borrows 
heavily from the work of Dr. Paul LeMahieu, currently Hawaii Superintendent of 
Education. LeMahieu challenges schools to move beyond the practice of merely 
“accounting” to the public about school performance and to understand “authen- 
tic accountability” as the act of taking responsible action to improve perfor- 
mance. In the belief that not only educators, but also parents and the community 
as a whole must take steps to assure that all students achieve to high standards, 
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LeMahieu urges schools to conduct public engagement processes designed to 
“ construct deep and commonly held understandings about the performance of 
the system, the reasons for the performance, the best possible ways of improving 
performance, and the responsibilities of each in doing so” (LeMahieu 1996) (see 
Figure 1). Taking up this challenge, schools and districts in WAC’s Kysosei 1 pro- 
ject have conducted over 25 dialogue sessions to date. 
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It is important to note that we are reporting our work with Accountability 
Dialogues in midcourse; at this writing the impact of these conversations must be 
considered promising, but incomplete. Our goal at this juncture is to provide 
school, district, and community leaders with a vision of what it looks like to 
bring the accountability conversation to the local level, and, in doing so, to cre- 
ate what we believe can become a powerful vehicle for generating continuous 
school improvement. 



2 WAC at WestEd designed and conducted professional development on implementation of 
standards-based reform and Accountability Dialogues for BASRC schools. 
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Why 

Accountability 

Dialogues? 



While accountability is becoming a fact of life for schools, some 

serious questions need to be addressed about the' validity of typical accountabili- 
ty measures and their usefulness in improving the educational system. 

Because accountability is both so important and so often misunderstood, schools 
and school communities often need a way to communicate about focal needs, to 
clarify understandings about how school and student progress will be assessed, 
and to agree about how to proceed. Over the past five years, Accountability Dia- 
logues have been used by some schools to do just this. 
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he current accountability movement has brought a new and much needed 
J sense of urgency to the demand that America’s schools meet the needs of all 
students. In the last five years, twenty-seven states have instituted new account- 
ability systems (Quality Counts 1999). 

Although accountability plans differ from state to state and district to district, 
most rely heavily on scores from large-scale student achievement tests, adminis- 
tered once a year, to determine school quality. Some factor in additional indica- 
tors such as dropout and attendance rates or teacher qualifications. In some 
plans, the consequences of inadequate performance are borne by individual stu- 
dents who are retained at grade level or not permitted to graduate; in others, 
principals lose their job or see their entire staff “reconstituted” in transfers to and 
from other schools. Some plans include consequences for students and adminis- 
trators alike. Still others provide incentives or rewards to schools or teachers who 
produce exceptional improvement. Some plans require, and fund, specific inter- 
ventions for individuals or schools not performing adequately, while others 
make no provisions for support. 

Critics of these accountability plans typically cite one or more of the follow- 
ing arguments in opposition to them: 



• They are not based on a commonly accepted definition of what a good 
school is; 

• They weigh too heavily the scores from a single assessment; 

• They fail to acknowledge that student performance is influenced by fac- 
tors stemming from outside the school; 

• They do not account for the time and expertise needed to build capacity 
in the system to address certain kinds of problems; 
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• They rely only on external accountability mechanisms; and 

• A number of unintended consequences may hami rather than help students. 

A Question of Quality: 

What is a good school? 



he pervasiveness of the call for accountability might lead one to assume that a 
J high degree of clarity exists about what we are looking for in “accountable” 
schools. Yet the often heated debate about accountability plans suggests that no 
such clarity exists. Should a school be judged by the degree to which students 
and parents are satisfied with their school experience? Or by the degree to which 
a school’s graduates succeed in the world beyond school? Or by some other kind 
of measure? 

Although there seems to be general agreement that the quality of student 
performance is an important indicator of a good school, there is little agreement 
about how such performance should be judged. Should the quality of a school’s 
or individual student’s performance be represented in relation to a fixed stan- 
dard? in terms of the degree of progress toward that standard? in terms of a rank- 
ing in relation to all other students or schools? in relation to students and schools 
with similar needs and similar resources serving similar student populations? 
And, if all of these should be taken into consideration, how should they be 
weighted to determine the quality of school performance? 

The range of answers to these questions reveals ideological differences 
about the purposes and goals of schooling that confound any easy attempt to 
provide an absolute definition of a “quality” school. Yet, without a shared under- 
standing of what quality means, it is difficult to hold anyone accountable for it. 

A Question of Measures: 

What data should be used to determine quality? 

ne of the loudest criticisms of most state- and district-level accountability sys- 
tems involves their over-reliance on a single means of measuring student per- 
formance — which usually means large-scale, standardized tests. These measures 
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are valued by the general public for providing reliable and objective compar- 
isons of students across schools, districts, and states. But educators often com- 
plain that these tests, given only once a year, provide an inaccurate or incom- 
plete picture of student performance. Numerous opponents claim the tests are 
biased, favoring students of certain cultural or linguistic backgrounds. Moreover, 
the results of these tests are reported in terms of rankings — thereby guarantee- 
ing that some students and schools will always be at the bottom. 

For the last two decades, educators and researchers have made many 
attempts to invent new assessments designed to provide a richer and more accu- 
rate picture of student performance. They envision, and in some cases have devel- 
oped, assessments that measure performance in relation to absolute standards and 
ask students to demonstrate that they can solve complex problems like those they 
will face in the world outside school. Many have argued that these measures 
should replace, or at least be used in conjunction with, traditional testing programs. 

A Question of Responsibility: 

Just who is responsible for improving student performance? 



'he call for increased school accountability raises questions about the degree 
to which schools can actually influence student performance and, therefore, 
should be held accountable for that performance. Researchers have continually 
noted that the strongest predictors of student achievement are non-school factors 
such as the level of parents’ education and income, the size and stability of the 
family, and the wealth and stability of the community. Many educators have long 
complained that they should not be held accountable for student performance 
when so many factors that influence it lie outside their control. 

Additionally, accountability plans are often criticized for not building in 
consequences for all parties whose work influences student achievement. Teach- 
ers wonder why parents are not held more accountable for supporting student 
learning, and how students might be held more accountable for the role they 
play in their own learning. Principals, whose jobs are on the line in certain 
accountability systems, cry foul when district leaders are not held similarly 
responsible for making schools successful. 
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A Question of Capacity: 

Do schools have what they need to make improvements? 




ome focus their criticism of new accountability plans on what they see as a 
minimal investment in building the capacity of the. system to improve: 



We see accountability plans that include plenty of consequences for 
poor performance and little if any help to improve performance.... 
They seem, to be prem ised on the idea that the problem is school profes- 
sionals ivho knoiu ivhat to do, but refuse to do it. Tloat is cenainly not 
our expenence (Tucker 1998). 



While some cheer top-down accountability systems for their strong stance, 
others criticize these carrot-and-stick approaches for what they see as a failure to 
recognize and provide what’s needed in terms of time, resources, and expertise 
to make the required improvements. 



A Question of Motivation: 

Can commitment be mandated? 



1 [Inderlying the current accountability frenzy is a quiet but restless debate about 
ci/what motivates schools and the individuals in them to change their practice 
and adopt successful strategies. Proponents of strong external accountability sys- 
tems generally believe that external mandates themselves bring about improve- 
ments. Yet many others believe that developing an internal sense of responsibili- 
ty for improvement is essential to the success of any accountability system. 

Michael Kirst, a professor at Stanford University and Co-Director of Policy 
Analysis for California Education (PACE), points out that many different state-level 
accountability systems have been tried and abandoned with little lasting impact on 
student performance. The reason, he suggests, has to do with a radical misreading 
by the creators of such systems — usually policymakers far removed from the 
classroom — of what motivates educators to improve their practice. Much research 
indicates that teachers are motivated not by external rewards or sanctions, but by 
satisfaction in seeing their students succeed at doing meaningful work. Conse- 
quently, says Kirst, if an accountability system has any chance of working, it 
must “build internal accountability that will match external accountability.” 
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A Question of Consequences: 

Will these new systems actually benefit students 
and increase student learning? 



F inally, countering the public enthusiasm for new accountability systems is a 
Greek chorus warning of their unintended and potentially counterproductive 
consequences. Commentators argue that systems developed by state policymak- 
ers may “shift the educator’s focus from serving the needs of students to serving 
the will of the state legislature” (Schmoker 1999). The results, they warn, will 
include a narrowing of the curriculum to those skills measured on state tests, a 
dangerous sameness to the learning opportunities teachers offer, and a mass exo- 
dus of dedicated, creative teachers from the profession. Where some foresee that 
accountability systems will galvanize schoolwide efforts to improve performance, 
others predict that they will make it increasingly difficult to find excellent educa- 
tors to work in the schools and with the students who need them most. 

The most vehement criticism comes from those who fear that the conse- 
quences of these systems will be visited unfairly upon students: 

The standards movement is a strategy which uses kids as burnable 
matter in the process of improving things.... The whole spirit of the pre- 
sent standards-and-test game is the spirit of top down: “We know what 
to do, here it is, you guys do it, and we'll give you a test. ” ... And it 
really hurts when the kids haven't had a chance to learn what's com- 
ing (Robelen 2000). 

Little data are yet available about how these various new accountability sys- 
tems will play out for teachers or for students. Nonetheless, the accountability 
movement rushes ahead with considerable public support. 
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to some, the term “Accountability Dialogues” is an oxymoron. Typically, 
accountability is seen as a system that affixes blame and initiates action from 
the top down. The school board calls the superintendent on the carpet for low 
test scores; the superintendent mandates new expectations and targets for the 
improvement of schools; principals plan a course of action that makes new 
demands on teachers; and teachers turn to students with increased emphasis on 
the areas identified for improvement. 

By contrast, Accountability Dialogues are designed to foster community- 
wide conversations that welcome the voices of individuals at all levels of the sys- 
tem — teachers, administrators, students, parents, employers, representatives 
from institutions of higher education, community leaders — and provide them 
with opportunities to share diverse perspectives and proposals (see Figure 2). 
Together they construct deep and shared understandings of the system’s perfor- 
mance, the usually complex reasons for that performance, and the various ways 
in which that performance might be improved. The goal is to develop a sense of 
shared responsibility and collective action, to generate accountability without 
blame (see Figure 3). 

By drawing on a multiplicity of perspectives and sources of data, Account- 
ability Dialogues acknowledge, rather than ignore, each of the challenges 
raised by large-scale, external accountability systems. The aim is not to simplify 
the complicated issues surrounding student performance, but to engage stake- 
holders, including the students themselves, in examining those issues. Account- 
ability Dialogues do this by creating a community forum that focuses on the 
work of the school, while acknowledging that the success of that school is 
enhanced by the support of those within the larger school community. These 
dialogues are a mechanism for joint problem solving aimed at moving the 
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school community away from superficial solutions and toward a culture of 
continuous improvement. 

Accountability Dialogues make possible a positive response to a set of high- 
ly charged issues: 

The current political climate creates enormous pressure for politicians 
and school leaders to define accountability in its most derogatory sense 
by laying blame and taking quick, strident, and highly public actions. 

Such actions are perhaps a tempting catharsis. But, although strong 
changes are often needed within the organization, lasting accountabil- 
ity cannot be achieved from the bully pulpit.... Improving performance 
and responsiveness within an organization requires cooperation and 
problem solving. The first step in solving problems is identifying them. 

When the central concern is finding individuals to blame, no one will 
bring problems forward, making it all but impossible to find construc- 
tive solutions to even the most basic issues (Tacheny 1999). 

Accountability Dialogues bring the accountability conversation home from 
the state capital to the faculty room or school auditorium. They acknowledge 
that accountability is a value developed and nurtured through relationships. 
They recognize that while measurement and management systems are the tools 
of accountability, the ultimate success of any accountability system depends on 
individuals whose daily practice is motivated by a personal commitment to their 
students, their colleagues, and their community. 
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What 

Schools Are Learning 
about 

Accountability 

Dialogues 



Accountability Dialogues propel efforts to implement standards- 
based reforms and at the same time strengthen the relationships among par- 
ents, educators, and the community at large. But the benefits to schools do not 
come without challenges. Hosting these dialogues requires careful planning 
and a good deal of time from already busy educators. Fostering new relation- 
ships with the community sends educators into uncharted territory. Schools 
look to a variety of strategies for promoting participation among diverse con- 
stituents and for assuring that the time spent pays off in a stonger program and 
improved student performance. 
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n ccountability Dialogues are designed to create a public and professional 
Ln uforum through which a school community determines what quality of perfor- 
mance it will require of its students, how that level of performance will be deter- 
mined, and what actions might be taken, and by whom, to improve student 
achievement. 

Through Accountability Dialogues, schools have 

• negotiated agreements on what students should know and be able to do; 

• built strong assessment systems and established performance standards; 

• come to better understand the conditions that contribute to or inhibit 
student achievement; and, 

• increased their sense of urgency and efficacy to improve student 
achievement. 

Establishing Agreements about 

What Students Should Know and Be Able to Do 

'TPypically, states establish standards by convening educators and content area 
J experts who are charged with the task of determining what students should 
know and be able to do in a given subject area. The resulting lists of “content 
standards” are approved by the state board of education and distributed for use 
by districts and schools statewide. These content standards are intended to serve 
as the backbone for the development of a state assessment system designed to 
measure school and student progress on these same skills. 

Not surprisingly, these lists are often found problematic at the local level. 
Because local educators usually have had little or no input into the creation of 
these standards, they feel little sense of commitment to them. Often educators 
and community members criticize the vague terminology or “educationese” of 
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the state-developed standards. Complaints from all quarters characterize these 
documents as wordy laundry lists of anything anyone might consider worth 
knowing — and as failing to emphasize those things that are really essential. 

Accountability Dialogues operate on the assumption that in order for educa- 
tors to teach effectively to standards, and for the community and parents to sup- 
port them, the standards must be understood and adopted at the local level. Dia- 
logues bring educators and community members to the table to forge agreements 
regarding what they want all students to know and be able to do. 

The standards developed in Accountability Dialogues acknowledge exter- 
nal guidelines about what is important for students, but at the same time reflect 
negotiated local values. State standards and those developed by professional 
organizations become valued resources used to balance local interests with 
externally validated points of view. For example, participants in Accountability 
Dialogues in one district examined the state mathematics standards related to 
problem solving and decided to expand on those standards for their own 
students. 

One magnet-school community used its standards-setting process to com- 
. municate its interest in and commitment to arts education. When the state and 
district provided content standards only in academic areas, the school convened 
staff members, together with professional artists, parents, students, and commu- 
nity members, to construct standards for music, drama, and the visual arts. 
Teachers wrote descriptions of how students would make progress toward these 
standards and then shared the drafts for feedback at an Accountability Dialogue. 

In another school, a dialogue that focused on understanding state test 
scores raised issues about important writing skills that were not being tested. 
Parents and teachers reviewed state test results and expressed concern that these 
scores gave them no information about students’ abilities to compose coherent 
narratives and reports, and gave students no information about how to assess 
the quality of their own work. As a result of the dialogue, school efforts focused 
on developing a writing assessment program and on working with parents to 
communicate the importance of them helping their children develop self- 
assessment skills. 
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Parents as “Critical Friends”? 



P arents conducting classroom observations? 0 

What’s that all about? At Montair Elementary jj 

School in the San Ramon (CA) Unified 0 

School District it wasn’t about “teacher shop- D 

ping.” It was about being “critical friends.” As the g 

result of the school’s Accountability Dialogue, 0 

parents were invited to visit classrooms in order jj 

to collect data and give the faculty feedback on q 

the consistency with which standards were being D 

implemented throughout the school. 

Montair has held a series of seven Account- 0 

ability Dialogues looking at the school’s writing jj 

program. Early sessions looked at the district’s q 

standards for writing and the alignment of the 0 

school’s assessments with those standards. jj 

Together parents and teachers studied data from D 

both norm-referenced state tests and the district’s D 
.. 0 

direct writing assessment. “We just weren’t satis- g 

fied with the results on the state test,” reported 0 
one parent. “When compared with students jj 

nationally, Montair students always do well. But □ 
in order to meet our standards in writing, we D 

want to know that kids can actually write some- g 

thing.” Students brought refreshing perspective to 0 
the conversation, noting that some tests go 
beyond measuring learning and actually promote o 
it. “I like the SAT-9 because it’s easy,” one child 0 
reported to the room full of parents and teachers, g 
“but on the writing test, my teacher can show me 0 
what I did wrong.” jj 

“We as parents just want to know what is g 

expected and what we can do to help our kids,” 0 
one parent explained. “The dialogues helped us jj 
a lot there. We got to see the rubrics, look at 0 

papers at the various levels, and think about, J 
Does my kid’s work look like this?” p 

But the data suggested that not all students 0 
were equally prepared to write well. Prompted jj 

by the facilitator at one dialogue session, parent, Q 

0 

0 

D 



student, and teacher participants offered a set of 
hunches about why that might be true: 

• Maybe some teachers do not use the 
language of the rubric to communicate 
expectations to students. 

• Maybe some students come to their present 
grade with very little experience in writing. 

• Maybe some teachers interpret the rubric dif- 
ferently than others. 

• Maybe some teachers give students more 
opportunities to practice writing than others. 

• Maybe some parents are not able to help their 
child at home. 

Together, dialogue participants decided to 
address their hunches in two ways. In order to 
investigate consistency across classrooms, partici- 
pants agreed that they needed information from 
observers who could visit many classrooms and 
talk to students. The faculty agreed to allow par- 
ents into their classrooms as data collectors. Par- 
ticipating parents agreed not to identify specific 
teachers by name, but to report on whether stu- 
dents were familiar with the rubric, whether 
scores seemed consistent across classrooms, and 
how much writing students did in class. 

To attack the problem on the home front, par- 
ent participants asked for strategies about how to 
help their children work on writing at home. Par- 
ents and staff agreed to collaborate on the design 
and dissemination of a newsletter that illustrated 
ways parents could promote good writing and 
work with students at home. 

“This is such a new way of thinking about 
working with parents,” said Susan Buck- 
Gordan, Montair’s principal. “I never would have 
believed the teachers would go for it. But I think 
the dialogues have helped us build a spirit of 
trust and commitment to get better. It’s been 
about, What can we all do?” 
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Through local school dialogues, the community often comes to recognize 
the importance of assuring that both students and parents understand the stan- 
dards well. One principal reported, “One of the things we decided in the dia- 
logues is that we as a staff were going to translate the standards into 'kid talk’ so 
that students would have a clear understanding of what they needed to know 
and how they would demonstrate their knowledge of it.” A teacher at another 
school noted that one of the goals emerging from the dialogues was to “make 
the targets [i.e., the standards] explicit to kids. If the target was something they 
always had in mind, then they could reach it.” As a result of parent feedback at 
an Accountability Dialogue in one district, the district developed a parent version 
of the district standards, carefully ridding the text of education jargon. 



Building Strong Assessment Systems 
and Establishing Performance Standards 



arge-scale state assessments are designed to reliably measure how schools 
and students are doing in comparison to one another. They are usually given 
once a year and are designed to cover a subset of the content standards 
approved at the state level. These tests generally rely on multiple-choice formats 
that are quick and relatively inexpensive to score but that do not assess students’ 
ability to compose their own responses to problems. Data provided by these 
assessments are not available to the school until months after the test is adminis- 
tered. And parents and teachers alike complain that the data are reported in con- 
fusing formats using terminology that is difficult to understand. 

In order to enrich the picture of performance provided by state-level stan- 
dardized tests, educators often use a variety of classroom assessments or other 
diagnostic performance measures designed to measure achievement in relation 
to content and performance standards. These measures offer rich and timely 
information to teachers and schools about what students can do with what they 
are learning, but they are often criticized for providing unreliable analyses across 
groups of students from different classrooms or schools. 

Accountability Dialogues have helped catalyze the development of assess- 
ment systems that provide both credible snapshots of how students are doing in 
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comparison to one another, and richer, more complex diagnostic information 
that tells what students can do and how they need to improve. Several schools 
have used their dialogues to present data from a variety of student performance 
measures, and to discuss the purposes, strengths, and limitations of each (see 
Figure 4). Parents talk with educators about why they want to know how chil- 
dren are doing in relation to one another. Educators illustrate how they use diag- 
nostic measures to guide instructional planning. Out of the conversation comes 
recognition on all sides of the need for a range of measures and a willingness to 
honor and invest in meeting the assessment needs of all those whose efforts con- 
tribute to improved student performance. 

Dialogues have also driven improvements in the reliability of classroom 
measures. As one teacher said, “The dialogue showed us that staff development 
around assessment was the missing piece. So we began to have discussions such 
as whether a three on my rubric means the same as a three on yours. We also 
came around to making formal agreements about how we were going to admin- 
ister and evaluate [classroom] assessments.” 

In another district, an Accountability Dialogue focused on how teachers 
and schools should weight different assessment results in establishing the per- 
formance standards for a given subject area. Discussion focused on the value 
that should be given to once-a-year state assessments and to those given at the 
district or classroom level several times a year. The community established per- 
formance standards in reading, writing, and mathematics that drew on a vari- 
ety of measures, including both those that provide highly reliable comparative 
data and those that provide timely diagnostic information at the classroom 
level. 

Increasing Understanding of the Conditions 
That Contribute to or Inhibit Student Achievement 

In an Accountability Dialogue, the community brings together multiple sources 
Jof data and multiple perspectives for the purpose of understanding not only 
how many students are or are not achieving to standard, but also the conditions 
that may be contributing to that performance. 

orr 
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Assessment data should be credible and useful. The following questions can 
help to evaluate any assessments that provide student performance data. 

Validity 

Does the information come from a procedure (or set of procedures) that 
actually measures what it sets out to measure? 

Reliability 

Is the information trustworthy? 

Fairness 

Does the information result from procedures that are appropriate for 
people of all backgrounds and needs? 

Usefulness 

Are the data useful to those decisionmakers who need it to do their 
work well? 

Appropriateness of Use 

Are the data used fairly and appropriately? 



Dialogues are often used to examine student achievement patterns that may 
be masked by summary reports providing only aggregate or average scores. At one 
school, for instance, participants discovered that while schoolwide achievement 
was adequate, achievement levels dropped significantly in the fourth grade. In the 
dialogue, participants generated hypotheses about possible causes. Perhaps fourth 
grade teachers — many of whom were new — were less familiar with the reading 
program than were other teachers. Some parents asked if there was adequate par- 
ent involvement at the fourth grade level and if the teachers needed more parental 
support to improve achievement levels. The dialogue led to a spirited but produc- 
tive exchange of views. “I valued the opportunity to really discuss what is happen- 
ing in important areas of student achievement,” one parent said. 

In more than one school, low overall scores in reading resulted from 
extremely low scores in one aspect of reading comprehension. “The breakdown 
of reading comprehension [as it was examined in the dialogue] showed me the 
areas I need to address with my students,” declared a middle school teacher. In 
another school, similar discoveries in the data surfaced concerns about the 
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e Power of the Informed 



D espite a significant investment of time and 
energy in developing a set of districtwicle 
performance standards and an assessment 
system to support them, a new majority on the 
district’s Board of Trustees threatened to disre- 
gard six previous years of work! 

The strategic plan developed by the San 
Mateo-Foster (CA) City School District in 1994 
had called for the development of a set of assess- 
ments to be used, along with the state assessment 
system, to determine whether students were 
achieving to grade-level standards. District lead- 
ers consulted with experts and convened teach- 
ers to investigate a number of assessment 
options. Determined to draw on both norm-refer- 
enced measures as well as assessments that pro- 
vide strong diagnostic information to classroom 
teachers, „ the district crafted a set of measures and 
reliable scoring processes for each. They then 
hosted districtwide Accountability Dialogues in 
order to explain the rationale for use of each 
instalment and to get input on how the scores 
might be combined to determine grade-level per- 
formance standards. Approximately 100 parents, 
teachers, administrators, and community mem- 
bers attended each of three four-hour sessions. 

Meanwhile the district and school site leaders 
conducted professional development sessions to 
assure that all teachers learned how to adminis- 
ter each assessment and to use the results to 
plan instruction and improvements in the school 
program. Parent meetings were conducted at 
each school to answer questions about how the 
assessment results would be used for individual 
students. 

But in the fall of the 1999 school year, some 
board members demanded school-level achieve- 
ment targets focused solely on the state’s norm- 
referenced test. District leaders feared that 



0 overemphasis on a one-time measure — and one 
jj that was not well aligned to the district’s stan- 
D darcls, to boot — would undo their many years 
0 of effort to build a richer assessment system. 
n The board room was overflowing with par- 
0 ents and teachers at both sessions set aside to 
jj discuss the school-level achievement targets. A 
0 teacher offered a simple metaphor to explain 
0 what he had learned by participating in the 
U Accountability Dialogues: “When a coach takes 
0 over a new ball club, he doesn’t just look at their 

^ win-loss record and scream, ‘Work harder!’ He 

0 

0 reads statistics, he looks at film, and he figures 
0 out what kind of help they need. That’s the idea 
U behind diagnostic kinds of assessments.” Later a 
D parent rose to represent a number of other par- 
jj ents who had attended the Accountability Dia- 
g logue sessions. “We think it is irresponsible,” she 
D read, “to ignore the input of the many parents 
jj and staff who participated in the Accountability 
D Dialogue events, who informed themselves 
D about the purposes and uses of assessment, and 
g who gave input to the design of districtwide per- 
Q formance standards.” 

jj By the end of the evening, the public out- 
0 pouring had an impact on the board, and they 
0 agreed to a set of achievement targets that drew 
g on both the state assessment system and the 

0 locally developed student performance standards. 
D 
D 
0 
0 
D 
D 
0 
D 
D 
0 
0 
D 
D 
0 
D 
D 
D 
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scarcity of reading materials available and prompted admissions that teachers 
lacked expertise in teaching some of the more technical aspects of reading. The 
dialogue resulted in specific actions to purchase the appropriate materials and to 
provide focused professional development for the teachers. 

Student involvement in the dialogues can also provide key insights. At 
one high school, for example, there was intense concern over low scores on 
the state-mandated standardized test. At an Accountability Dialogue, students 
speculated that test scores were lower than expected because many students 
believed that the test was of no consequence to them and had therefore not 
taken it seriously. Parents and teachers alike used these comments as a jump- 
ing off point to examine how they communicated with students about the test 
and its purpose, and to consider the ways in which they might motivate stu- 
dents to do their best. 

Increasing a Sense of Urgency 

and Efficacy to Improve Student Achievement 

nnnhen faced with troublesome patterns of low student performance, school 
VI VI personnel can often feel overwhelmed. Not surprisingly, some succumb to 
blaming forces outside the school — “the district,” “parents,” “society,” or the stu- 
dents themselves. These schools can become paralyzed by their low expecta- 
tions of themselves and of their students. The variety of perspectives and sense 
of shared responsibility nurtured through Accountability Dialogues create not 
only a sense of urgency for improvement, but, equally important, an awareness 
that working together increases everyone’s chance of success. 

In one high school dialogue, attention focused on the concern that 40 per- 
cent of all freshmen had GPA’s under 2.0. When additional data revealed that 
many of these students were reading at three to four grade levels below that of 
the texts used in freshmen classes, plans were set in motion to work with the mid- 
dle and elementary schools to strengthen articulation of standards for student 
performance at all grade levels. In addition, participants called for more immedi- 
ate action to address the needs of the high school’s current students. As a result, 
the school made plans to purchase more materials at appropriate reading levels 
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What If You Gave an Accountability 
Dialogue and Nobody Came? 



W ilson Elementary School’s first Accountabili- D 

ty Dialogue was held on a Saturday, in [j 

hopes of making it easier for working par- □ 

ents of the San Leandro (CA) Unified School Dis- D 

0 

trict to attend. Despite the fact that the principal y 

and a small group of staff members had planned 0 

the event carefully, they were all a bit afraid of j! 

what might happen. The data they had to share 0 

did not paint a rosy picture of student perfor- jj 

mance, and they were genuinely concerned y 

about how parents would react to it. 0 

They needn’t have worried. Hardly anyone jj 

showed up. Discouraged but unwilling to give up, D 

the planners rethought what it might take to get jj 

parents to attend such an event. And most impor- y 

tantly, they talked about how to encourage the 0 

involvement of a variety of parents, especially 

those whose children were doing poorly in D 

school. ^ 

0 

The planners decided to ask each teacher to y 
target five parents with a personal invitation to 
attend. They encouraged the teachers to call (] 

those parents with whom they had had the least D 
contact all year, and the principal supplied teach- jj 
ers with a script that they could use to describe q 
the event. Wilson’s second try at an Accountabili- & 
ty Dialogue was scheduled on a Wednesday 

evening, and flyers were sent home in English, D 

0 

0 

0 



Spanish, and Cantonese inviting parents to an 
event called “Parents and Teachers Together: An 
Honest Look at Wilson’s Reading Scores and 
Other Information about Our Students and 
School.” On the second try, the Wilson community 
drew 140 participants, including a large percent- 
age of the staff. Asked why they voluntarily 
attended, staff members admitted that they 
could not afford to miss out. Said one teacher, 
“When so many parents expressed interest in 
coming, we knew we had to be there!” 

What hadn’t changed — to anyone’s satisfac- 
tion — was the student performance data. Teach- 
ers, parents, district leaders, and school board 
members studied information showing how stu- 
dents performed on the state test. They also 
looked hard at other data, such as attendance, 
discipline patterns, homework return rates, fre- 
quency of reading at home, and parent satisfac- 
tion with various aspects of the school program. 
Together participants speculated about what 
needed to be done to improve student perfor- 
mance. Parents suggested the need to organize 
volunteers to read with students on a regular 
basis, expressed concern about the low atten- 
dance and homework return rates, and asked for 
more information about what was expected of 
students. Teachers expressed the need for better 



and to participate in professional development designed to help teachers learn to 
teach reading skills through their content areas. 

A similar story emerged from an elementary school where a look at student 
performance data at a year-end Accountability Dialogue revealed unacceptably 
low reading achievement. When some participants suggested that the school 
begin an investigation of available professional development opportunities, oth- 
ers voiced their impatience and called for action that would “get something up 
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assessments and clearer performance standards, 0 

and for opportunities to learn better strategies for jj 
teaching reading. 0 

Spurred by this well attended dialogue, the 
Wilson staff addressed parents’ needs for better g 
communication in the following ways: 0 

• Staff shared the expectations for student perfor- [j 



mance with parents at Back to School Night; o 
• An information hotline was set up for parents jj 
in English, Spanish, and Cantonese, and addi- g 
tional funds were allocated to provide transla- 0 
tors at all school events and at parent confer- jj 
ences when needed; 0 



• Student performance data were shared at par- 
ent conferences and the urgency of attending g 
to low performance was emphasized; and D 

• A new program was instituted to encourage 

parents and family members to read together 0 
on a daily basis. |j 

Also as a result of input from the dialogue, the g 

staff agreed to reallocate resources to target low 0 

performing students: 

• Students with the worst attendance and/or D 

tardiness records were identified and staff jj 

were assigned to communicate with those stu- g 
dents’ families on a daily basis and to target 0 
counseling and social services to those fa mi- 

lies as needed; 0 

D 

0 

0 



• Monthly meetings of support staff were insti- 
tuted to assure that students participating in 
more than one of these programs receive 
coordinated services; 

• The school’s Title I program was restructured 
so that students would no longer be pulled 
out of their regular classrooms, but would 
instead receive extra support before and after 
school; and 

• Services to English learners were restructured 
by grouping students with similar levels of 
language development. 

Following the dialogue session, every partici- 
pant was sent a summary of the event and a list 
of how the questions raised there were being 
addressed. 

Wilson’s persistence in the effort to engage 
large numbers of diverse representatives from 
their school community continues to pay off. 

An Accountability Dialogue in the fall of 1999 
brought out another 130 participants. Free child- 
care, the addition of a Korean translator, and 
plenty of snacks to fuel hungry participants con- 
tributed to the evening’s success. Looking back 
on the event, one key planner commented, 
“Planning this dialogue was like planning a fami- 
ly wedding — just as complex, and just as 
rewarding!” 



and running by the start of the new school year.’ 7 Working from the energy creat- 
ed in the dialogue session, the school contracted with experts from a local uni- 
versity and, with their help, designed a year-long professional development plan 
in reading that involved weekly modeling of effective, teaching strategies, indi- 
vidual teacher coaching, and all-staff training sessions each week. 

At this writing, student achievement at Wilson Elementary School is on the 
rise, although staff remain frustrated with their inability to close the achievement 
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gap between middle-class students and those who are low-income, English 
learners, or students of color. While no one is claiming that Wilson’s promising 
gains are a result of the school’s Accountability Dialogues, these events nonethe- 
less have motivated the staff to persist in their improvement efforts. Cindy 
Cathey, Wilson’s principal, comments: “We have an amazing, committed and 
dedicated staff. The Accountability Dialogue helps them reflect on their practice 
and refocus on what they can do differently to close the achievement gap.” 

In one rural school district, an Accountability Dialogue served as a forum 
for a group of typically disenfranchised parents to share their interests and 
needs. Spanish-speaking parents in great numbers used the session to communi- 
cate their sense of urgency in seeing their children learn English, their desire to 
learn English themselves, and their need for more information about how they 
could better help their children at home. District leaders who might otherwise 
have, passed up a grant opportunity to fund after-school programs immediately 
applied for those funds. As a result, an evening program for parents and their 
children was established. At this writing, 50 families have participated, with 
60 more on a waiting list. 

In sum, Accountability Dialogues have played an important role in pro- 
pelling the work of school improvement. They have provided a forum through 
which administrators, teachers, parents, community members, and students come 
together, often for the first time, to look deeply and purposefully at how the 
school is doing and what can be done to improve it. Stories abound about the 
ways in which these conversations have generated or strengthened important 
commitments, provided common learning experiences, or guided action at the 
school level. 
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t the same time that Accountability Dialogues accelerate the implementation 
lof standards and facilitate the use of data to plan improvements, they also 
upset the traditional ways in which educators have related to parents, the com- 
munity at large, and one another 

Parent, educator, and community participants report that Accountability Dia- 
logues offer several benefits: 



important new roles for parents; 



increased understanding of the diversity of perspectives about school 
quality; 



• opportunities for open and honest communication; and 

• increased sense of shared responsibility and demand from within. 



Important New Roles 
for Parents 



n many public schools, parent involvement has largely been confined to fund 
Jraising and boosterism. Yet more and more parents are demanding to be 
involved in substantive educational issues. 



There is a growing disconnect between the thin layer of the nation ’s 
experts, professionals, and leaders, and the general public. Attempts by 
leaders to sell their viewpoint to a public that has not experienced the 
same information, discussion, or debate are unlikely to succeed . 
Today's public will not blindly follow what experts propose; they need 
to experience the opinion-formation for themselves. 



Any successful parental involvement thrust will require a more direct 
engagement between the school and the home — one that takes into 
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consideration the values of the family and of the community (Dykstra 
and Fege 1997). 

Calling on educators to involve parents and the public in “the intrinsically 
worthy debate over what knowledge, what skills, and what sensibilities we might 
want to nurture in the young,” Dykstra and Fege warn that in failing to do so 
“we will lose not only parental engagement, but with it, the ‘public’ in public 
schools.” 

Accountability Dialogues foster meaningful and direct engagement of par- 
ents in issues that reach the core of their children’s educational experience. 
Increasing numbers of parents have demonstrated great eagerness to participate 
in this way. Asked at the close of each dialogue session whether they would be 
willing or interested in participating in similar sessions in the future, well over 
90 percent of participating parents said they would. Attendance figures have 
demonstrated their sincerity. Every school and district has reported increasing 
levels of attendance with each subsequent dialogue. 

Educators participating in Accountability Dialogues have come away with 
new understandings about the ways in which parents can contribute to a 
school’s efforts to improve student performance. They report being impressed 
not only by how concerned parents are with student achievement, but by how 
interested they are to understand the complex factors that influence perfor- 
mance. Educators have also discovered that parents are eager to embrace their 
own role in increasing student performance. One school leader remarked, “The 
most important thing that came out of the dialogue is our partnership with the 
parents and the community. They want to celebrate our successes with us, but 
they push us with hard questions, too.” 

Increased Awareness 
of Diverse Perspectives 

y bringing together stakeholders with diverse viewpoints, Accountability 
Dialogues help participants realize that different stakeholders have very dif- 
ferent and equally strongly held perspectives about what a good school is and 
how their school can be improved. Drawing on this variety of perspectives and 
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a variety of data, participants come to understand that issues regarding school 
and student performance are far more complex than any one person or group 
had previously thought. 

Accountability Dialogues provide a forum for exploring difficult issues such 
as those underlying the gap in achievement levels between white middle-class 
students and low-income students or students of color. In a dialogue, many per- 
spectives about the root of the problem are shared. Are there differences in the 
quality of instruction offered to different groups of students? Has one group had 
better access to an effective intervention program? Are there cultural differences 
in the way these children experience school? Do teachers have access to cultur- 
ally relevant curricula? Are some parents less able to provide academic support 
to their children at home? Do all teachers have the materials they need to teach 
well? Participants who may have arrived at the session with a single hypothesis 
about the cause of the problem come to understand that a variety of factors may 
be in play and that a number of actions may need to be taken. As one parent 
reported, “I learned that many different things need to be analyzed when evalu- 
ating what’s going on at our school.” 

Participants report that the more data a school shares, the more they come 
to understand what factors and conditions may contribute to student success. “In 
addition to test scores, we showed attendance figures, tardiness figures, suspen- 
sion rates,' and more,” said one organizer of an elementary school dialogue. 
Added a parent, “We are now asking the right questions in order to get a richer 
picture of what goes on for kids. It helps us take a proactive stance. If we remain 
committed to using data to drive instruction, we will continue to make strong 
growth in student achievement.” 

New Norms of Honest 
and Open Conversation 

"O^oth educators and parents report going into their first dialogue with trepida- 
-QJtion. Parents’ worries include “not knowing what it would be about” or “not 
knowing if I would understand what they are talking about.” Educators acknowl- 
edge being reluctant to share information that they do not fully understand or that 
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they have traditionally kept to themselves. As one high school teacher allowed, “I 
really wasn’t looking forward to this. The thought of the faculty being in a fish- 
bowl with a lot of parents around looking at our work scared me to death.” 



tive nature of the dialogue. Parents report being extremely appreciative of the 
school’s willingness not only to display the data, but to openly discuss what 
those data say about how well the school is addressing issues of student achieve- 
ment. Indeed, even when the data reveal areas of concern, participants have 
responded not with anger but with an attitude of What can we do about it? One 
elementary school teacher remarked, “There was a fear that if we shared the 
data, parents would say, ‘Do you know what you’re doing?’ But it didn’t happen. 
Instead it felt like a true team effort — we were all in it together for the sake of 



yet to be tested by time, for now, at least, a school’s willingness to open itself up 
to public scrutiny has been met with appreciation and support from the larger 
community. 



A Sense of Shared Responsibility 
and Demand from Within 

hile top-down accountability mechanisms rely on mandates, coercion, and 



compliance, an Accountability Dialogue strives to generate momentum 
from within the system to improve itself. School leaders and community partici- 
pants in Accountability Dialogues report that “our pronouns are beginning to 
change.” The focus of conversation shifts from what “they” need to do to what 
actions “we” need to take. 

The dialogues generate what might be called a sense of “lateral” account- 
ability: teachers challenge colleagues to change their practice; parents question 
other parents about what they can do to better support student learning. In one 
school where reading achievement was lower than expected, teachers them- 
selves decided to disaggregate scores by classrooms in order to determine if 
teachers participating in a professional development pilot showed a greater 



Time and again, however, participants have been surprised by the produc- 



tive kids. 



Although it is true that this spirit of openness and collective goodwill has 
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effect on student learning than others. A parent at another school with similarly 
troubling reading scores called on fellow parents to set up a volunteer program 
through which adults would come to the school on a regular basis to read to stu- 
dents and listen to them read. 

In one districtwide dialogue that extended over a series of meetings, atten- 
tion focused on the realization that the graduating eighth graders were the entire 
community’s “final exam.” In other words, whether students achieved satisfactori- 
ly in eighth grade was a reflection not only of the middle school, but also a reflec- 
tion of the efforts of elementary feeder schools, parents, students, and the larger 
community. After attending the dialogue held at the middle school, elementary 
school staffs and parents reported returning to their own sites with a renewed 
sense of commitment to preparing all students to succeed in middle school. 

Increased internal or lateral accountability may also increase the capacity of 
schools to deal in productive ways with the demands of external accountability 
mechanisms. 

Internal accountability includes the norms by ivhich teachers oper- 
ate , the expectations they hold about student learning and their role 
in improving it } and the processes they use to carry out their ivork . In 
schools ivith iveak internal accountability, the norms emphasize the 
individual responsibility of each teacher over student learning, 
rather than the collective responsibility of the entire school. In those 
cases, teachers' judgments about ivh ether and hoiv much they could 
improve student learning depend on their understanding of the stu- 
dents' background and lack a perspective of ivhat students could do 
under different circumstances. 

Similarly, the expectations for student learning in such schools are rel- 
atively loiv, since teachers believe that the conditions the student 
brought to school, rather than their own efforts, exert the greatest influ- 
ence over their academic performance. Teachers in schools ivith loiv 
internal accountability tend to place a greater emphasis on order, an 
expectation each teacher shares. 
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Schools with weak internal accountability therefore tend to respond to 
external pressures for change by summoning their own individual 
beliefs, rather than by consulting with colleagues and attempting to 
work collectively for improvement (Elmore and Rothman 1999). 

Several school leaders have described their experiences with Accountabili- 
ty Dialogues as transforming events. Says one, “Our Accountability Dialogues 
have been the highlight of my educational career. So many people in one room 
struggling together over what matters for children. Sometimes I despair that we 
won’t find the answers we need, and then I get re-energized by the level of 
commitment of the people in the room.” Another states that she can no longer 
imagine being a principal in a school that would be unwilling to open itself up 
to host similar dialogues. 
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"F^espite the benefits schools experience from Accountability Dialogues, veterans 
UJof planning and conducting these meetings acknowledge the very real chal- 
lenges they pose. Challenges that can be anticipated when conducting Account- 
ability Dialogues are often the flip side of the benefits yielded by this new 
approach to school improvement. Planners find that they must 

• acknowledge new roles and relationships; 

• encourage diverse participation; 

• amplify the voice of all participants; and 

• integrate dialogue into the ongoing work of the school. 

See Appendix A for a checklist to help plan and conduct these dialogues. 

Acknowledge New Roles 
and Relationships 

“por educators, parents, and the public alike, inviting all stakeholders to identify 
J and solve problems through dialogue is new. Assumptions prevail that 
accountability can only be realized through top-down mechanisms. Educators 
have little experience working with their larger community to identify and solve 
problems; and parents have little experience working on substantive issues of 
school improvement. 

In successful Accountability Dialogues, facilitation of the agenda and the 
nature of the conversation are not left to chance. Planners allow time in each ses- 
sion to assure that all participants know why they are there,' what the school hopes 
to achieve at this and any future sessions, and how the input gathered from partic- 
ipants will be used. (Examples of overhead transparencies used to introduce the 
concept and purpose of Accountability Dialogues are included in Appendix B.) 
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To help the group communicate in productive ways, facilitators often take 
time to involve everyone in setting norms for the nature of the conversation. They 
then ask all participants to be guardians of those norms, by both practicing them 
and by reminding the group when they are not being used. The norm-setting 
process models the school’s commitment to using the dialogue to hear from all 
interested stakeholders. At the same time, it teaches strategies for making all par- 
ticipants responsible for the success of the session (see Figure 5). 

Planners also generally provide a balance of role-alike and cross-role group 
activities within each session. Time is made available for teachers to talk with 
other teachers and for parents and community members to talk together — as 
well as for conversations that include representatives of a variety of role groups. 
It is not uncommon for the role-alike conversations to lead to an awareness that, 
despite their similar vantage point, everyone in a given role-group may not think 
alike — or, in cross-role groups, to discover that there is more agreement across 
roles than was previously assumed. 

Encourage Diverse 
Participation 

(nichools report that some of the greatest challenges of Accountability Dialogues 
(yJinvolve ensuring broad participation in what is, at first, a new and unfamiliar 
activity. And even schools that succeed in recanting diverse stakeholders still face 
the challenge of assuring that the information shared and the issues discussed are 
presented in such a way that all participants can contribute productively. 

Several schools have found that the term “Accountability Dialogue” is off- 
putting to potential participants and have changed the name of the event itself, 
using titles such as “A Community Conversation,” “Through a Parent’s Eyes,” or 
“An Honest Look at Student Performance.” Still others hold onto the term 
because they feel it symbolizes their commitment to accountability and draws 
participants who share that interest. 

Most schools’ first sessions drew few parents or other community members. 
Those who did attend were generally parents already involved in school 
activities. Even staff participation was often limited to the principal and a few key 
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Respecting Others 

✓ Respect various perspectives 

^ Replace “but” with “however” and “and” 

%/ First seek to understand and then to be understood 
i/ Listen as an ally 

✓ Don’t personalize comments — address ideas, not people 
i/ Operate on a level of caring acceptance 

Taking Responsibility 

✓ Don’t assume prior knowledge 

✓ Allow time to think — allow for backtracking 

i/ Offer constructive suggestions about the process 

✓ Make goals explicit and monitor the group’s progress 
^ Take individual responsibility for the group’s success 
i/ Set and respect time limits 

i/ Resist overgeneralizations like “all” and “always” 

1 / Confidentiality 

^ Provide evidence to support assertions:What’s the evidence to support that? 

Good Communication 

^ Be honest and direct — not so “nice” that nothing gets done 
^ Avoid jargon and “educationese” 

^ Try not to be overly sensitive 

^ Listen to each other — leave titles and roles at the door 
^ Be nonjudgmental 
^ Check for understanding 
1 / Summarize as you go along 

*From one school’s Accountability Dialogue 



planners. But over time, these same schools found powerful ways to increase 
participation of both community members and staff. In one school, teachers 
agreed each to call the five parents with whom they felt they had the least con- 
tact during the school year and to encourage their participation. As a result, well 
over 100 parents attended the next event. When teachers realized that the 
Accountability Dialogue was a valuable opportunity to hear from the community 
in new ways, they voluntarily increased their participation. This school has 
consistently engaged over 100 community members in an annual dialogue and 
attendance has become an expectation for all staff. 
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At one high school, low parent turnout for the first Accountability Dialogue 
was initially explained away as the unwillingness of parents to participate in 
school events. But, building on feedback from the few parents who did attend 
that first session, the school targeted the invitations to the next dialogue to par- 
ents of the school’s newest students and even those in feeder schools. Letters 
were sent to parents of ninth graders (and of eighth graders from the feeder 
schools) inviting them to join “a conversation about what will be expected of 
your student, how the school program prepares students to meet our standards, 
and what you can do to help assure their success.” After retargeting the invita- 
tion, the school now struggles to accommodate the large number of parents and 
students who attend the dialogues. 

Increased public engagement in communities comprising families of diverse 
socioeconomic, cultural, and linguistic backgrounds has brought its own chal- 
lenges. Planners have realized the need to provide translators at the sessions — 
one dialogue was conducted simultaneously in English, Spanish, Cantonese, and 
Korean. Other schools have tried holding, immediately before the more general 
dialogue session, “affinity group” meetings involving parents and community 
members representing similar language minority groups or those sharing com- 
mon interests in a specific issue. 

Because the topics discussed at Accountability Dialogues are new to many 
participants, and often require some background knowledge that may not be 
shared by all, planners frequently include activities that help the different role 
groups develop a common understanding of key concepts or terms. For exam- 
ple, many dialogues require all participants to understand the many purposes for 
assessing student performance and that different assessment tools each have var- 
ious strengths and weaknesses in meeting these needs. Accordingly, facilitators 
have designed activities that teach the relevant but specialized concepts of valid- 
ity and reliability. Participants can then draw on these new understandings in dis- 
cussing the purpose and quality of the various assessments used to judge student 
performance in their district or school. (See materials in Appendix C that schools 
have used to introduce key assessment concepts.) 
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In opening up to a larger public questions of “how students are doing,” 
Accountability Dialogues have revealed schools’ general lack of capacity to 
gather, manipulate, and use data. As participants learned to ask increasingly 
sophisticated questions about the conditions that might contribute to improv- 
ing school performance, they have often been frustrated by the school’s 
inability to produce data to help answer those questions, When data were 
available, they were often presented in cumbersome or incomprehensible for- 
mats. Partially as a result of their efforts to conduct powerful Accountability 
Dialogues, nearly all schools have invested in new school-level data manage- 
ment and display software, and they are pressuring their districts to develop 
districtwide data information systems designed to accommodate the complex 
questions that can arise from dialogue sessions. 

Amplify the Voice 
of All Participants 

(n^chools conducting Accountability Dialogues experience a tension in balancing 
[yJthe goal of broad participation with the goal of using the dialogue to plan 
targeted action for improvement. Planners are distinctly aware that the time partic- 
ipants can give to dialogues is limited and precious — and that the more people 
in attendance, the more difficult it is both to hear from them and to draw on all 
their different ideas to plan action. Yet limiting participation in any way risks cut- 
ting off the contributions of some stakeholders who may have a unique perspec- 
tive on school quality or how to improve it. 

Schools have addressed this challenge in different ways. In several schools 
each table group at a session is assigned a facilitator who is charged with taking 
notes from the small group’s discussion. Notes are typed up and distributed to all 
session participants for comment before the next session or are reviewed by the 
faculty in subsequent planning sessions. Many dialogues have made use of indi- 
vidual “Participant Logs,” in which participants are asked to record their thoughts 
and reactions to the discussion periodically throughout the session. Participant 
Logs are anonymous, but comments are identified as having come from a parent, 
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community member, or educator. Participant comments from logs are summa- 
rized by role group, shared, and used to plan subsequent sessions. Some schools 
survey or conduct focus groups among various stakeholder groups about the 
issues to be examined in an upcoming dialogue, and then bring the data from 
these investigations to the dialogue. In this way the voices of teachers, students, 
parents, and community members who cannot attend sessions are “heard” and 
can be considered by those present. 

Integrate Dialogue into 
the Ongoing Work of the School 

hile planners and participants in Accountability Dialogues almost univer- 



sally express enthusiasm for the impact these events have had on their 
school and its work, it is obvious that the future of these dialogues is, at best, 
tenuous. Planners have often expressed concern about the demands and 
complexities of planning them, recognizing that the time they spend doing 
so is time away from other duties or instructional responsibilities. As one plan- 
ner reports, “We’ve been astounded by the parent interest and concern the dia- 
logues have generated. But make no mistake about it, a successful dialogue is 
not easy to pull off. It takes us the better part of a week to put one together. It 
takes a lot of time to gather data and to think through the best ways to present 
it. And then there is the need to re-present information from the earlier dia- 
logues [when new participants attend a subsequent session].” 

It becomes increasingly clear to those involved in these sessions that in 
order to perpetuate what seems to be a promising mechanism for accountabili- 
ty, these dialogues must find their way into the routines of school life. They 
must replace other less powerful activities rather than be “tacked-on” to 
already overly demanding school schedules. Some schools have considered 
replacing typical site advisory group agendas or site council agendas with 
agendas similar to those used in Accountability Dialogues. One district that was 
committed to a districtwide strategic planning process used site-level dialogues 
to launch it. 




43 



Accountability Dialogues: School Communities Creating Demand from Within 



POStDBB @ 




Using Accountability Dialogues, schools can anticipate the following 
outcomes over time: 

%/ Increased Understanding of Standards 

%/ Increased Accessibility to Good Data 

^ Increased, More Diverse Participation 

%/ Increasingly Helpful Questions 

%/ Increasingly Focused Action for Improvement 

%/ Increased Satisfaction with the Pace and Progress of Improvement 



The true measure of the success of Accountability Dialogues will be when 
they are no longer scheduled as isolated events — when inclusive dialogue 
about school quality and how to improve becomes a norm of professional and 
public practice (see Figure 6). 
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What Do Dialogues Add 

to the Current Focus on Accountability? 



)0®DD 



F or those who have experienced them, Accountability Dialogues embody 
many of the characteristics commonly associated with accountability. They 
promote transparency — the willingness of the school to be open about what it 
values and how it functions. They demand disclosure — the reporting or 
accounting of achievement results. And at least in some cases, they have resulted 
in targeted action to improve student achievement. 

Yet for some, no doubt, this mechanism runs counter to notions of “real” 
accountability. To the degree that these dialogues cloud a sense of who is 
responsible for school quality and soften the consequences placed on individu- 
als, they will leave some unsatisfied. Still others will dismiss them as too time 
consuming and complex; simpler notions of accountability — such as manage- 
ment by a system of “rational” and tightly coupled standards, measurements, and 
consequences — will unquestionably be more attractive. 

But studies have shown that external systems of rewards and sanctions may 
do little to improve student performance in the long run, and that low-achieving 
school districts are perhaps least able to respond to typical accountability 
approaches (Macpherson 1998). In one study, for example, the culture of low- 
achieving school districts was found to be “resistant to the use of coercive power, 
lacking in culture-changing capacities, and, sadly, encouraged [by traditional 
accountability mechanisms] not to develop self-transformative capacities” (Gins- 
berg and Berry 1998). 

When accountability measures were seen by these schools as failing to 
acknowledge the complex factors that influence student achievement, school 
personnel responded with “denial, selective inattention, aggression, even exam- 
ples of arrogant disregard of legitimate interests and deceptive manipulation.” 
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Additionally, because these were districts with no history of improvement, their 
low student achievement, represented by “rational” data, was experienced as a 
permanent condition, not as a challenge. In fact, the researchers conclude, there 
is a “real possibility that the politics of accountability in these school districts are 
part of the problem of low achievement.” In schools like these, Accountability 
Dialogues may be especially important to ameliorate some of the unintended 
and even counterproductive effects of external, top-down accountability systems. 

An Accountability Dialogue is a mechanism that acknowledges the neces- 
sary reciprocity of accountability and capacity. If a person with formal authority 
requires that someone or some group be held accountable for an action or out- . 
come, then the person in authority has an equal and complementary responsibil- 
ity to assure that the responsible party has the capacity to do what is asked 
(Elmore 2000). Designed to encourage conversation both within and across role 
groups, Accountability Dialogues provide a forum through which parents learn 
to communicate to teachers about the support they need to assist their children. 
Teachers talk to parents about the support they need from home and to adminis- 
trators about the training, materials, or other resources they need to accomplish 
schoolwide goals. Cross-role coalitions grow out of the opportunity provided to 
share perspectives and needs. 

Accountability Dialogues acknowledge that a range of complex factors 
influence school performance but, at the same time, provide a context for con- 
fronting excuses. Well facilitated, they require professionals and community 
members alike to look critically at evidence and to listen carefully to a variety of 
perspectives about why things may be as they are. Powerful Accountability Dia- 
logues increase the school’s capacity to improve student achievement, both by 
drawing on a variety of perspectives to identify problems clearly and by building 
a sense of shared responsibility for addressing them. 

The greatest promise of Accountability Dialogues lies in their potential to 
strengthen the relationships that are essential to sustained school improvement. 
Educators demand more of each other; parents expect more from their peers. 
Barriers of language and experience among educators, parents, and the public at 
large begin to fade. Perhaps most importantly, through dialogue, accountability is 
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operationalized not only as a set of external management systems but as a 
shared value or cultural norm of the school and larger community. 

Accountability Dialogues are no panacea. They take time and care to orga- 
nize, and initially they throw professional educators and the public into roles 
they admit to feeling ill-prepared to accept. But our experience over the past 
five years suggests that the Accountability Dialogue is a promising mechanism 
that both acknowledges the complexities of improving student performance and 
promotes productive responses to very legitimate calls for school improvement. 



0 

ERIC 



47 



Conclusion 49 




Dykstra, J., & Fege, A. F. (March 19, 1997). Not without parents. Education Week, 
16 (25). 

Elmore, R. F. (Winter, 2000). Building a new structure for school leadership. 
Washington, D C.: The Albert Shanker Institute. 

Elmore, R. F., & Rothman, R. (Eds.). (1999). Testing, teaching, and learning: 
A guide for states and school districts. Washington, D.C.: National Academy Press. 

Ginsberg, R., & Berry, B. (1998). The capability for enhancing accountability. In 
Macpherson (Ed.), The politics of accountability: Educative and international 
perspectives (pp. 43-61). Thousand Oaks, CA: Corwin Press, Inc. 

Kaku, R. (July-August, 1997). The path of kyosei. Harvard Business Review, 
55-63. 

LeMahieu, P. G. (1996). From authentic assessment to authentic accountability. 
Delaware: Delaware Department of Public Instruction. 

Macpherson, R. J. S. (Ed.). (1998). The politics of accountability: Educative and 
international perspectives (pp. 43-51). Thousand Oaks, CA: Corwin Press, Inc. 

Quality counts: Rewarding results, punishing failure (special issue). (January, 
1999). Education Week, 18 (17). 

Robelen, E. W. (January 12, 2000). Educators honor ‘Doc’ Howe’s contributions. 
Education Week, 19 (17), 23-25- 

Schmoker, M. (November 3, 1999). The quiet revolution in achievement. Educa- 
tion Week, 19 (10), 33, 35. 

Tacheny, S. (March, 1999). If we build it, will they come? Educational Leadership, 
56 (6), 62-65. 

Tucker, M. (Spring, 1998). Beyond accountability systems that punish. The 
Newsletter on Standards-Based Reform, 1 (2). 




48 



Checklist for Planning and 
Conducting Accountability Dialogues 





v 





49 

d 

ERIC 



Appendix A 53 



Checklist for Planning and 
Conducting Accountability Dialogues 



Schools that have experience with Accountability Dialogues offer the following 
tips for planning and conducting these events. 

Setting a Context 
for Potential Participants 

✓ The purpose of the Accountability Dialogue is to get input from the larger 
school community about how the school is doing, why things are the way 
they are, and how they might be improved. When announcing an initial 
Accountability Dialogue, plan far enough into the future to let participants 
know how this dialogue fits into a larger picture, what the next steps will be, 
and when future dialogues will take place. 

What Topics Will You Address 
in Your Accountability Dialogue? 

When planning a dialogue, decide what outcomes you hope to achieve. 
Keep these to a manageable list and be realistic about how much you can 
learn about, discuss, and agree upon each time people come together. 

^ You may want to. accomplish your goals in small steps. The first dialogue 
might be simply to inform people about the process, provide participants with 
a “state of the school” report, and generate interest in continued participation. 

*/ Often, early dialogues are used to establish a set of standards for school and 
student performance. 

✓ Once the standards are set, later dialogues might explore how the school is 
doing in relation to these standards and what concerns the community has 
about that level of achievement. Ideally, the dialogue provides a place for 
participants to share perspectives and possible explanations for the school’s 
strengths and weaknesses and to offer suggestions about what actions might 
be taken to improve. 

Dialogue sessions have focused on questions such as the following: 

• What do we think students in our school should know and be able to do? 
Are these the same as or different from our state content standards? 

• Is a high but realistic performance standard? 

• What examples of student work best exemplify our performance standard? 

• Are the assessments we use to measure adequate? If not, what 




50 

Appendix A 55 



additional information do we need or how would we revise our assess- 
ment procedures to give a better picture? 

• What do our data tell us about how our students are performing in 
? 

• What hunches do we have about the factors that contribute to our pat- 
terns of weakness or to our school’s strengths? What data might we want 
to collect to check out those hunches? 

• What intervention strategies might we invest in to improve school perfor- 
mance? What is the role of each stakeholder group in the success of that 
intervention? How will we know if it works? 

1/ Remember that dialogue is two-way; it is not merely to inform, gain 
approval, or rubber-stamp decisions already made. Decide what tasks really 
do need to be accomplished through deep, open dialogue as a community 
and be prepared to listen. 

How Will You Present 
and Discuss Data? 

\/ Encourage participants to generate questions about school or student perfor- 
mance that they would like to see answered. Early dialogue sessions might, 
for example, begin with the question, “What data would you like to see in 
order to understand whether this is a good school?” 

%/ In later sessions you may want to encourage participants to generate ques- 
tions about what is working and not working at the school. For example, a 
parent might speculate that students seem to do well until the third grade, but 
then “fall off the track.” If enough parents share that speculation, you will 
want to provide data that help the group find out whether that hunch is true. 

✓ Know your data well. The facilitator, staff, and other planners should have 
examined the data in advance in order to effectively point out strengths and 
areas of concern and to facilitate the participants’ discussion of the data. 

Assessment data can be notoriously difficult to understand, so present data as 
clearly as possible: 

• Summarize in advance. 

• Provide necessary background (assessments used, when administered, 
etc.). 

• Simplify data displays (charts and graphs). 

• Make sure data are clearly formatted, labeled, explained, etc. 

Assist participants to understand trends that influence the data, including 
demographic changes. 
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✓ All data have limitations, therefore, you will want to create space for partici- 
pants to raise questions and doubts about the data. 

✓ Separate data-reading from data-interpretation, analysis, and action planning. 
Be sure that your facilitator is prepared to correct faulty or hastily drawn 
conclusions. 

Who Will Participate 

and How Will They Be Invited? 

Make a list of various “stakeholders” in your school community — any group 
that has an interest in school or student performance and/or plays a role in 
improving it. Consider what might motivate each of these groups to partici- 
pate and develop an invitation strategy that addresses those interests. Let 
people know the nature of their participation, how much time is involved, 
and what benefits they will receive. 

In order to encourage the participation of community members and of par- 
ents who have been reluctant to participate in school events, work with com- 
munity leaders to reach those with whom they have contact. 

Consider how staff will be represented at the event. In some dialogues they 
have played a listening, clarifying, and recording role while other members 
of the community did the talking. In other cases, where a good portion of the 
staff could attend, they participated fully, expressing their own opinions and 
concerns. Do not expect one or two teachers to “speak for the staff.” If staff 
cannot attend in significant numbers, poll them in advance and bring reliable 
data about staff opinions to the dialogue session. 

^ Involve students whenever possible. 

^ Be sure to follow up dialogues with information to participants about what 
was accomplished because of their involvement, and what remains to be 
done. 

How Will You Build the Capacity 
of Those Who Attend to Participate Fully? 

/ At the start of each session, establish norms or agreements about how partic- 
ipants will communicate their ideas with each other. Ask everyone in the 
group to take responsibility for following the norms throughout the session. 

•/ Plan to use some part of an early session to generate ideas about what par- 
ticipants hope to get out of their participation: Is it a desire to help their own 
children? To learn more about the school? To add their voices to decisions? 
To express their concerns about what the school is doing? Acknowledge 
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these various motivations and let participants know how or if those interests 
will be addressed. 

1/ Consider what individuals need to know in order to participate fully and 
appropriately, and make plans to address those needs. For example, parent 
and community participants may need a glossary of educational terms or a 
chance to see how an assessment or survey is administered. Everyone might 
need a chance to practice reading data tables together before being left on 
their own to discuss what the data' say. It might be wise to seat participants at 
tables where a variety of role groups are represented; in this way, expertise 
can be shared from one group to another. 

•/ Provide translations of all materials and/or translators for language minority 
groups. 

Arrange for table groups or other small-group formats so that everyone has a 
chance to speak and to learn from one another. Time may not allow for small 
groups to record or report out key issues from their discussions, so provide 
participants with logs in which they can record individual thoughts, opinions, 
or questions. Be sure to review these logs between sessions and, in your 
planning, try to acknowledge issues raised. 

^ Small things make a big difference. Plan for refreshments, good lighting, ade- 
quate ventilation, convenient restrooms, comfortable seating and room tem- 
perature, availability of parking or public transportation, and child care. 

What Materials and Equipment 
Need to Be Prepared? 

Arrange for plenty of clerical/administrative support — beforehand, on the 
day of the dialogue, and to follow up with documentation and further com- 
munication. 

Have handy all reference documents you might need. If everyone will be 
using the same materials, make sure you have sufficient copies for'every par- 
ticipant. Excerpt or provide summaries of long documents, as needed. 

Wall charts and/or projected transparencies help all participants focus on 
similar information. In large rooms, it is best for the facilitator to talk from 
these large documents or projections while participants look at copies. 
Microphones and a good sound system are a must in large rooms. 

e/ If you are using videotapes that require people to listen closely, be sure the 
sound quality is excellent, pause the tape to repeat a point, or provide listen- 
ers with a transcript of the tape. 
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Who Will Facilitate 
Your Dialogues, and How? 

✓ Dialogues can be facilitated by anyone with the time and skill to do so. 
Although many schools assign the principal this task, others ask a person 
from the district office, a parent, or a community member to guide the group 
through its work. A facilitator should have experience conducting large 
group discussions and the ability to encourage participation, resolve conflict, 
build consensus, and guide decisionmaking. It is the facilitator's job to 
remind the group of the norms they established for conversation and to keep 
the whole group on task. 

✓ Planners should work with the facilitator to create an agenda that uses time 
effectively. Establish and honor clear starting times, breaks, and ending times. 
Include essential elements such as appropriate welcomes, warm-ups, engag- 
ing activities, chances for feedback, a sense of closure, and a statement of 
next steps. Make sure the facilitator lets people know what will be done with 
the ideas generated at the dialogue and how they can keep track of progress. 

✓ When anticipating large numbers of participants, you may want to assign 
table leaders who can assist in leading small-group discussions, going over 
materials, debriefing activities, reaching decisions, etc. Plan an orientation 
session for them in advance of the dialogue so that they understand their 
responsibilities. 

✓ Document the event — with videotape, sign-in sheets, evaluation forms, etc. 
Make these available to those who are interested but could not attend. 

✓ Soon after any dialogue, convene the planners to reflect on how it went, go 
over the outcomes and what was accomplished, and note ideas for 
improvement. Keep a journal, portfolio, or other record of what you do, 
what you learn, and to reflect on your progress. 
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Overheads to Introduce Concepts 
of Accountability and Accountability Dialogues* 




Permission to copy these overheads is hereby granted. The Western Assessment 
Collaborative at WestEd invites you to use them in conducting Accountability 
Dialogues. 



Accountability Dialo gues 

From Accounting to Accountability 

Assessment for 

ACCOUNTING 

“Accounting refers to the act of gathering, organizing, 
and making available for use any of a variety of 
information describing the performance of the system.” 

Assessment for 

ACCOUNTABILITY 

“Accountability takes that information and uses it to 
inform judgments about performance and how it can be 
improved — including planning and action.” 



ACCOUNTABILITY DIALOGUES 

“The goal is to involve various constituencies in the 
construction of deep and commonly held 
understandings about the performance of the system, 
the reasons for the performance, the best possible ways 
of improving performance, and the responsibilities of 
each in doing so.” 

- Paul G. LeMahieu 
“From Authentic Assessment to Authentic Accountability” 
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Accountability Dialogues 

Stakeholders 



Parents 




Support 

Staff 



© 2001 WestEd, Western Assessment Collaborative 



Accountability Dialogues 

New Ways to Think about Accountability 



Not ... 

Blame 

Reporting 

Hierarchical 

Only External 



But . . . 



Responsible Action 



Dialogue 



Lateral and Shared 



Also from Within 




© 2001 WestEd, Western Assessment Collaborative 
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Accountability Dialogues 

Anticipated Outcomes Over Time 



^ Increased Understanding of Standards 

^ Increased Accessibility to Good Data 

^ Increased, More Diverse Participation 

^ Increasingly Helpful Questions 

✓ Increasingly Focused Action for 
Improvement 

^ Increased Satisfaction with the Pace and 
Progress of Improvement 
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Materials for Assisting Participants to Look 
at Student Performance Data* 




* Permission to copy these materials is hereby granted. The Western Assessment 
Collaborative at WestEd invites you to use them in conducting Accountability 
Dialogues. 
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Accountability Dialogues 

How Good Are Our Data? 

Validity 

Does the information come from a procedure 
(or set of procedures) that actually measures what it 
intends to measure? 

Reliability 

Is the information trustworthy? 

Fairness 

Does the information result from procedures that are 
appropriate for people of all backgrounds and needs? 

Usefulness 

Are the data useful to those decisionmakers who need 
them to do their work well? 

Appropriate Use 

Are the data used fairly and appropriately? 



Adapted from “Moving Up to Complex Assessment Systems,” Proceedings from the 1996 CRESST 
Conference published in Evaluation Comment, a publication of UCLA's National Center for Research on 

Evaluation, Standards, and Student Testing 
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Crash Gordon Takes a Test 

Crash Gordon has been enrolled in Fly-by-Nite Pilot School for three weeks. The 
school promises that by successfully completing the course, Crash will be ready to pilot 
747 commercial jets. Crash has been told that Fly-by-Nite is a highly regarded school. Its 
ads report that nearly 95% of Fly-by-Nite graduates score above average on the final 
exam. The school is also known for its low tuition costs. Five years ago Fly-by-Nite 
replaced its expensive flight simulators with textbooks and films that explain in detail 
what anyone might want to know about flying a plane. 

Today is Crash’s mid-term exam. If he passes with a grade higher than 50 points 
he can skip the rest of the course and will get his license right away. If he scores from 
25-50 he will need to repeat the course, and if he doesn’t complete the exam or scores 
below 25, he will be kicked out of Fly-by-Nite. The stakes are high. 

Crash has no idea what will be on the test. He arrives a little early and is handed a 
computer disk on which, he is told, the questions have been recorded. Crash feels really 
lucky that he happens to have his laptop computer in the car, and he goes to get it. On his 
way back he meets several of his classmates who are leaving dejectedly. They have been 
told they cannot take the test without a computer. Among them is Crash’s good friend 
Yoshi, who has been flying 747s for the Japanese Air Force for 10 years. The battery in 
Yoshi’s computer is dead. Yoshi will flunk out of Fly-by-Night. Crash offers sympathy to his 
friend and then moves quickly to a seat near one of the few electrical outlets in the room. 

Crash boots up his exam and finds 100 multiple-choice questions which he 
needs to answer in 30 minutes. Approximately one-third of the questions are about the 
parts of a plane, another third are about how to read a flight schedule, and the final third 
cover various aspects of the dress code for pilots. None of these topics has been covered 
in the course. About one-half of the questions in each section are written in French or 
Spanish. Crash figures this is because 747s usually fly international routes. The final 
question asks for a brief written answer to the question, “Who has most influenced your 
life as an aviator? Explain.” 

Crash is not daunted by the test. As soon as he realizes that he will need to 
answer the items quickly, he develops a plan for filling in the answers according to a 
pattern. He decides that every fourth question will be “A,” every third question “B,” and so 
on. Knowing that his teacher, Mr. Soar, will be scoring the exam, Crash uses the brief 
essay to explain how much Soar’s teaching has meant to his career. Crash finishes the 
exam with three minutes to spare. 

When the exam scores are posted on the bulletin board at Fly-by-Nite, Crash is 
elated. His test-taking strategy has paid off. The roster shows that Crash got 29 points. He is 
especially pleased to note that the average score on the test was 27 — being above average 
gives Crash the confidence that he will do better next time. Crash leaves Fly-by-Nite eager 
to improve and become a pilot. He decides to go on to the airport and sit near the crew 
lounge for an hour or so. This way, he reasons, he’ll learn a lot about pilot dress code. 

© 2001 WestEd, Western Assessment Collaborative 
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Accountability Dialogues 

Using Crash Gordon to Think about 
the Quality of Our Own Assessments 



Think about and discuss: 

1 . What problems can you identify in terms of the validi- 
ty, reliability, fairness, usefulness, and appropriate use 
of the data generated by Crash’s assessment? 



2. Before looking at the data from the assessments your 
school is using, take time to consider each assessment 
according to the characteristics of good data. You may 
find that each has its own strengths and weaknesses. 
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Accountability Dialogues 

Key Questions to Structure 
Discussions about Data 



1 . What do these data tell us about 
performance at our school? 



2. What questions do we have about 
these data? 
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This book explains * 

some of the common misper- 
ceptions about school account- 
ability and provides a strong 
rationale for including 
Accountability Dialogues in 
any accountability system. 

Not only do Accountability 
Dialogues propel efforts to 
implement standards-based 
reforms, they also strengthen 
the relationships among 
parents, educators, and the 
community at large. 



“In all the talk about school accountability, we 

sometimes lose sight of the fact that, fundamentally, schools 

have to be accountable to parents and communities. 

Accountability Dialogues are a practical strategy to make this 

kind of accountability real.” 

— Merrill Vargo 

Executive Director 

Bay Area School Reform Collaborative (BASRC) 



“Clear and common sense thinking about this 

important educational issue that is all too often politicized 
beyond recognition.” 

— Terry Mazany 

Associate Superintendent 
Southfield Public School District 
Southfield, Michigan 



Stories and examples from 
schools that use Accountability 
Dialogues provide a real sense 
of what can happen when 
responsibility for school 
improvement is shared among 
all the stakeholders in a school 
community. 

This book and a companion 
videotape have been devel- 
oped from the work of the 
Western Assessment Collabora- 
tive at WestEd. 

J 



“Such important work to help school systems! 

The lasting result of Accountability Dialogues for schools and 
districts is increasing their capacity to move beyond the quick 
fix — to dig deeper and to take strategic action to make real 
change.” 

— Karen Schauer 

Curriculum Director 

Galt Jomt Union Elementary School District 
Galt, California 



“Accountability without a process of engaging 

stakeholders at the school level is an empty box. The Western 
Assessment Collaborative has gone a long way toward filling 
this space with a structured, humane, and reasoned approach. 
Accountability Dialogues avoid past pitfalls and enable 
practitioners and community members to see accountability 
using a common lens, vocabulary, and interests.” 

— Rudy Crew 

Director of the Institute for K-12 Leadership 
Former New York City Schools Chancellor 
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